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ICTRODUCTrOK 


Quadratic  proKramiain?  and  linear  coraplementarity  are  two  clor.ely  related 
mathematical  nrogramr.ing  ayiecialtits  having  otroni.  relevance  in  engineer- 
in,’'  plasticity,  as  well  as  otlier  seeniingly  distant  disciplines.  The  aim 
of  this  paper  is  to  provide  a coherent  int.-oluction  to  these  s’jhjecto.  Tne 
presentation  is  deliberately  focussed  on  fundarjentals , the  purpose  being 
to  conununicate  Che  basic  theoretical  and  algorithmic  concepts  and  to  indi- 
cate where  the  reader  may  turn  for  more  details. 

The  discussion  of  quadratic  programming  g'ven  here  is  essentially 
lirrdted  to  the  convex  case.  Fortunately,  this  covers  most  of  the  commonly 
encou-ntered  applications  of  QP.  The  development  includes  a presentation 
of  the  optimality  conditions  and  the  construction  of  dual  qua,lratlc  pro- 
grams. A few  of  the  many  available  convex  quadratic  prograran.irg  algorithms 
are  also  described. 


The  consideration  of  optimality  conditions  in  Q?  gives  rise  to  the 
formulation  of  the  linear  complementarity  proble.m.  The  TCP  is  a useful 
framework  within  which  the  QP  and  its  associated  algor’’ thmo  can  bo  studied. 
However,  the  LCP  arises  in  other  ways,  too.  Some  highlights  of  the  theory 
and  a selection  of  the  principal  computstiona'  ' iques  are  assembled  in 
this  paper. 

In  some  apclications , it  is  necessary  to  solve  a parametric  QP  or 
LCP.  Such  problems  are  discussed  here  from  the  tlieoretical  ard  computa- 
tional standpoints. 

Potential  users  of  QP  and  LCP  software  will  definitely  be  inter- 
ested in  what  can  be  expected  in  terms  of  computational  efficiency.  The 
available  information  on  thin  in  somewhat  scanty,  t'ieverthel ess , an  attempt 
has  been  made  to  summarize  what  has  been  rej'orted  in  the  literature. 

!^or  reasons  of  space,  the  proofs  of  the  theorems  quoted  here  are 
omitted,  but  they  can  be  found  in  the  cited  references.  As  for  specific 
engineering  plasticity  applicat lens , many  will  be  found  in  the  other  papers 
of  those  Proceedings,  so  there  would  seem  to  be  no  point  in  lengt.hening 
tills  article  bv  developing  them  again.  ■ 


a;.')  CKAr-'\TRY  of  QUAiWATir 


Quadratic  ii'  tnnt  part,  of  inalhcwatica]  j.roKrai'.L’idni’  • ^rjcernod 

with  the  ,T.i  nlini  ttation  (of  L'iaxir:ii i^atioti ) of  u quadratic  functJor.  Of  njany 
variables  subject  to  linear  con.jtvaints . A',  a rul'',  these  linear  con- 

straints include  inequalities.  Indeed,  fro:n  the  standpoint  of  theory,  it 
is  possible  to  convert  any  1 inenr  contraint .•  to  linear  inequalities  in 
nonnegative  variables.  Tiirougi.out  this  paper,  the  statement  of  the  quad- 
ratic programming  problem  will  be 

minimize  7x  + ^jcHx  (1) 


subject  to 


A X < b 


In  this  formulation, 

A G b e K®,  c e if , H G 

are  given;  the  quadratic  function 

Is  the  ob.^eotive  functi on.  and  the  set 

C = {xGif  1 Ax^b,  x>0} 

is  the  constraint  set  (or  feasible  .set).  One  can  assume  that  the  matrix  H 
appearing  in  the  definition  of  ^ is  symmetric,  for  othervdse,  it  cai  always 
be  replaced  by  its  s;,'Tr..metric  part,  h (H  + H_) . 

llie  constraint  set  C is  always  convex  and  polyhedral,  whereas  the 
objective  function  is  convex  on  if  ana  only  if  if  is  positive  semi- 
definite,  meaning 


X H x > 0 for 


e r". 


When  this  condition  obtains,  the  Od’  (l)  is  called  a cenvex  cu.ciratic  pro- 
rrr rr,.  If  K is  positive  definite,  so  that  (2)  holds  as  a strict  inequality 
for  nil  X 0,  the  function  ¥ is  .strictly  conve.-: . and  (l)  i.s  then  called 
a .strictly  convex  quadratic  program.  When  is  not  convex  on  f , its  re- 
striction to  C can  still  be  convex  if  that  set  is  of  lower  dimension  than 
n.  At  any  rate,  when  (2)  is  not  valid,  (l)  is  ccm.’nonly  called  a nonconvex 


Among  QP  problems,  the  convex  case  is  much  easier  to  deal  with 
both  theoretically  end  methodologically.  Fort'uiately , the  applications  of 
convex  ^ arc  sufficiently  widespread  to  m.oke  it  worthy  of  Independent  de- 
velopment. In  the  author's  opinion,  it  is  of  greater  importanee  to  be 
abl>'  to  solve  large  convex  quadratic  programming  problems  than  nonconvex 
problems  of  small  size.  Accordingly,  the  convexity  as. sumption  will  be 
ra' her  freely  invoked  throughout  tliif;  survey. 


A vector  x*  t C :iuch  thrit  <p  (x*)  < '_x)  ‘'jr  aYl  ,x  ^ C 1;:  sidii  to 

be  c-td  i rci ' for  (l). 

The  const  rtf  ■ r.t  ii  of  a QP  are  l:k<-  there  bn  LP,  itnd  wi.en  H.  = 0_, 

(l)  really  is  an  I.P.  It  is  a well  hiicwn  P'.ct  that  if  an  LP  har;  an  optimal 
solvition,  t’len  it  must  tiave  an  optir.al  EC'l'itlon  vi.ich  is  also  jii  extreme 
point  of  C.  In  this  respect,  a .-onvex  C,P  U~  ur.lik--  OiJ  I,P.  It  is  not 
necessary  for  its  optical  sol’atiorv.,  if  ar.j  , occ<a'  at  extreme  points  of 
C.  Simple  illustrations  ca.n  Le  giv-n  to  illustj-aV_-  tiiat  tliere  exist  con- 
vex QP  problems  with  interior  poir:t  soliitions,  boitndai-y  point  solutions, 
extreme  point  solution:  or  no  solutions.  Despite  tr.li:  major  diflerer.ee 
between  LP  and  QP,  thoi’e  are  elements  of  great  similarity  betweeri  the-  two 
classes  of  problems.  .Some  of  these  will  b'  emphasized  in  the  next  two 
sections . 

3.  OPTIMALITY  CONDITIOIJS  AND  DUALITY 

3.1.  Ontiir.ality  Condi  tions . In  all  branches  of  mathematical  pi-ogramming, 
it  is  essential  to  know  how  to  recogi.'ize  an  optimal,  solution.  In  QP,  this 
is  done  vdth  the  Kidin-Tucker  conditions  [3?j,i3l]  which  were  developea 
for  and  apply  to  nonlinear  programs  of  much  greater  generality.  The  pre- 
sentation cf  optimality  conditions  given  in  this  section  is  a spec  illza- 
tlo.n  of  "Kuhn-Tucker  theory"  to  the  QP  case. 

Tl'.e  following  result  expresses  the  necessary  conditions  of  opti- 
mality for  a QP  of  the  form  (l).  iTie  theorem  is  valid  regardless  of 
whether  the  objective  function  is  convex. 

Theorem  1.  If  r*  is  an  optimal  solution  to  the  QP  (l),  then  there  exists 


vector  y*  s>ich  that 


£ t X*  + 

X*(=  + H x»  -*■  A ^»)  = 0 
*y_*{b  - A 2c*)  - 0 


Equatiems  (3)  - (6)  are  called  ttie  Kul;n-Tucker  cond itions  for  (iV 
There  are  several  points  worth  noting  about  them.  First,  conditions  (it) 
and  (5)  are  knowm  as  the  complementar:/  slackness  conditions . Thc?y  have 
the  .‘'ollowing  significance.  .Cince 

= £ + ^ jt * -t  A >0 
and  )c*  ^0^,  the  equation  (it)  irapl.'es  that 

x*u*  = 0 J - 1,. .. ,n  (7) 


M'  rv  i C'  ’ 1 y 


r 


I 


i 

t 


X*  > C implioi.  ' 
i*  J 


J = 1 


u*  > 0 implic';  x*'  = 0 

X.'  V 

5ince  j<*  must  bo  feasible. 


V*  = b - A X*  >0. 


Thus  (5)  says 


and 


y*v»  = 0 
•'  1 1 


> 0 implies  = 0 

V?  > 0 implies  y*  = 0 
1 ' •’  1 


i = 1,. . . ,m 


i = 1, . . . ,m 


(8) 


(9) 


(10) 


It  does  not  follow,  however,  that  the  implications  (8)  ar.d  (10 ) can  be  re- 
versed. 


When  = (b  - ^ = 0,  the  i-t)i  constraint  is  said  to  be  bind- 

inp:  (or  active)  at  x-  Notice  that  if  jc*  ^ £ is  optimal  for  (l)  and  no 
constraints  are  bindin/:  at  x * i then  ~ 2.  Z.*  ~ P.'  that  x*  is  an 

interior  point  of  C,  and  the  optiinaiity  condition  asserts  the  familiar 
equation  = £ + ?i  = 0. 

Ihe  componer.ts  of  are  kno-wn  as  Lagranpe  multipliers  for  the 
coi-respondin^’:  constraints  h - ^ > 0^.  The  components  of  u*  are  Lapirange 

multipliers  for  the  constraints  ^ ^ 0_-  The  numbers  y?  can  be  interpreted 
as  measures  of  the  rate  of  change  of  the  minimum  value  of  the  objective 
fur.ction  ^ with  respect  to  changes  in  the  corresponding  components  of  b. 
Tliis  interpretation  is  worked  out.  in  f53].,[l^]. 

'Ihc  Kulin-Tucker  conditions  are  merely  necessary  conditions  of 
optimality,  ’’fa  feasible  vector  £ and  a ncnnegative  vector  y iiappen  to 
.satisfy  (3),  (*^),  and  (5),  tliere  i s no  guarantee  that  21  must  be  an  optimal 
solution  of  (l).  One  cannot  expect  too  much  from  first-order  inform.ation. 

Convexity  of  f i’equi valent ly , the  positive  semi-definitcnoss  of  i^) 
is  stcond-or.!'' r i nf'. real i ori  of  a type  that  guarantees  the  suf  fi ci  ency  of 
of  ■'.he  Kuhn-Tuf'kcr  conditions  as  an  optimality  criterion. 

Theorem  c.  let  (x)  = £ x 'i  £ " H he  convex  on  If  x*  is  a feasi- 

ble vector  for  (j  ) and  there  exists  a vector  y_*  such  that  (x^",y_*)  satis- 
fies (3),  (l»),  (5),  and  (6),  then  x*  is  an  optima '!  solution  for  (1). 

In  the  convex  QP  case,  the  Kuhn-Tucker  conditions  are  necessary 
and  sufficient  for  optimality.  The  scarcii  for  optimal  solutions  of  such  a 
prohlcm  can  be  confined  to  solutions  of  the  Kuiiti-Tucker  system, 

*V 

u = £ + 11  X + A 2?^  £ £ 0.  £ u = 0 (11) 

v = b-  A2t£0,  2.^.  0,  'y£=0 


I* 


i.?,.  St.vTiPi-  "i  .’ict.'ii.'lf’  ii;-  I ;-j,j  ;,o  rnr  ■■DOW,.  ' ic 

exi.rteuce  awl  uni  jj.  -oT  ■d;-.-,,  ; . i.;  . ' ; cf  the  i.-.-.in  rfs.iKs 

in  this  depari w.  ' 

Thicorort:  3.  2 r the  quadratic  I'un.-tion  bcunde!  iulcw  cii  tf.e  nonef.pty 

polyiiedrnl  convex  set  £,  tlien  V a’-t.nu'S  1'.'  on  C. 

To  the  author's  knov]  ■•.■d-w , ini'=^  result,  kirn/n  as  tl.e  Krank-Vo]  fe 
theorem,  Tirst  appeared  in  [2^*  ] . It  doe.;  not  reci'uire  the  conve.xity  ol’  ^ , 
but  rfhen  y IS  strictly  convex,  It  is  po-.tible  to  state  a sti-onger  tlieorem. 
Theorem  1.  A strictly  convex  Qi’  -x.ith  a nonempty  constraint  set  has  a 
■unique  optima.l  solut'ion. 

Although  the  uniqueness  of  the  optimal  solution  is  ciiaracteri Stic 
of  all  mathematical  progri-uns  ha^ving  strictly  convex  objective  functions 
and  convex  constraint  S'Cts,  the  existence  i;  not.  It  is  apparent  that  for 
a strictly  convex  Qp,  the  objective  furiction  ^ (x)  = £ + '2  £ fl  x is 
boundeci  below  on  r”  ’'and  hence  on  £)  by  ^(-K  ”'_c)  - 4 pH  ^ £•  Thus,  the 

existence  of  tlie  optimal  solution  is  assured  by  the  Frank-Wolfe  theore.-a. 
More  can  and  will  be  said  about  the  existence  of  sol'utions  in  the  sequel. 

3.3.  Duality.  Associated  with  a convex  QP  (l)  is  a maximization  problem 
called  its  dual.  The  dual  is  also  a QP  defined  ■ulth  the  same  data.  It  is 
customary  to  refer  to  (l)  as  the  primal  problem.  Thus,  for  the  primal 
problem 

minimi  ■/.a  ^ (x)  = £ x ’'2  x _H  x 

subject  to  H 2?  H 

X >.  0 

the  corresponding  dual  problem  is 

maxi.mize  X H X (12) 

Av/ 

subject  to  £ i(  X ■*  H i 2. 

y >.  2 

This  dual  probleia  involves  t’rfo  vector  variables;  x and  y_.  Only  y is  re- 
quired to  be  nonnegati VC  (although  it  is  known  thnt  nonnegnti''rii y can  be 
iiLp-cscd  on  X vithou''  ios;i  cf  generality). 

Theorem  3 . If  x.  feasible,  for  (l)  and  ( .x' ,y)  is  feasible  for  (12),  then 

"P  (21’ .Z.)  < V W (13) 

Moreover,  if  equality  holds  in  (13),  then  )c  and  (x',y_)  are  optimal  solu- 
ticiir,  cf  (1)  and  (12'i,  respectively. 

Tiiis  so-called  weak  dual  Ity  theorem  hnsj  .-.everal  impl  i cati  ons , one 
of  wlii'Th  i.s  that  if  the  primal,  and  its  duai  are  both  feasible,  they  must 
both  pe"srss  optimal  solutions — a coritetpicnce  of  the  Frank-Wolfe  theorem. 


5 


r 


However,  Lhe  Main  renuJ.t  in  thin  'lircction  in  Iiorn'j  dualily  Liic- jia.-m  [^1]. 
'I'heoretn  C.  I f sol  \/es  (l),  tlicri  there  exintn  a victor  sacii  that 
selves  (12),  and  = fix*). 

This  can  bo  seen  by  tahing  as  the  vector  of  lagran,;o  Sinlil pliers 
guaranteed  by  the  Kul'.n-Tucker  theoi’era.  By  perfonr.ing  suitable  ir.anipulu- 
tions,  it  is  possible  to  verif\'  the  "converse  duality  tkeorciTi" 
llieoreiu  T.  If  the  program  (l2)  has  an  optimal  solution,  then  so  does  (l), 
and  the  optimal  values  of  their  respective  object: Itmctions  are  equal. 

If  a QP  is  feasible  and  fails  to  have  an  optimal  solution  its  ob- 
jective function  must  be  unbounded  in  the  direction  of  extrerci zation.  In 
fact,  in  the  case  of  convex  QP,  the  unboundedness  of  the  objective  function 
of  one  problem  is  equivalent  to  the  infeasibility  of  its  dual. 

It  should  be  noted  that  Dorn's  duality  theory  of  convex  QP  gener- 
alizes that  of  DP  in  nearly  all  respects.  Howeve:',  the  structures  of  the 
paired  QP  problems  are  dissimlar . This  is  rectified  in  the  s^anmetric 
duality  theory  developed  by  Cottle  [8].  If  one  takes  the  primal  as 

minimize  fix,'/)  + (1^) 

subject  to 


X ^ 0 


then  the  dual  is 

maximize  f ix,/)  = - b^  - H^Hx  - ^ L S.  L (^5) 

subject  to 

il  i £ 

In  these  problems  called  symmetri c dual  quadratic  rronrams . both  H and 
are  assumed  to  be  positive  semi -defini te  and  syra.mctric.  Wnen  Q "=  £,  this 
pair  of  proble.ms  reduces  to  the  dual  pair  considered  by  Dorn.  The  duality 
theorems  quoted  above  are  also  valid  for  these  symmetric  dual  programs. 
Anotlser  result  is  called  the  .ioir.t  solution  theorem . 

TTieorem.  8.  If  ilh)  and  (15)  are  both  fc-as’ble  programs,  then  there  exists 
a pair  of  vectors  iy^  ,'jf)  which  solves  both  problems,  and  in  paj-ticolar , 
(j^*,v*)  yields  a solution  to  the  Kui;n-Tuckfr  system  (ll). 


I+.  QUADRATIC  PROGRAMIilNO  AT  GOP.ITHMS 


•i.l.  Introduction . Among  the  many  algorithms  specifically  designed  for 
solving  quadratic  programs,  three  are  preeminent.  Tliese  are  the  methods 
of  Beale  [l],  [2],  [3],  Wolfe  [58],  and  van  de  Panne  ajid  Whinston  [**7], 
(li8),  [50].  Each  is  a slraplicial  method  in  the  sense  that  it  involves  a 
sequence  of  pivot  steps  similar  to  those  of  the  simplex  method  of  DP. 
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ir 


}'ut,  uiilike  t'l'-  -si.".plex  methc'i  for  M.o  pivot  rtej  3 of  there*  QP 

alcoi'ithm;'  Jo  not  gcncr^to  a oei;uoncc  of  «.  ■ lj'.t.o  poi.'.tj  of  the 

priical  cOiirtnint  set  £.  Instead , they  work  with  a larger  rystetn  of  In- 
eiunlities.  In  effect,  they  are  concerned  witli  sat  i of:/!  ng  the  h'uhr.-Tuck.er 
cyst'.ra  (11)  for  the  primal  jiroiiJ.cm,  thoug’i  each  doc..-:  so  in  a different  way. 
It  is  known  that  if  tlio  system  (ll)  iias  a roiution,  then  there  exists  a 
solution  which  is  an  extreuo  point  of  the  set  of  points  Batisfjing  its 
inequality  constraints: 

U-c+lijc  + A^^O  (16; 

v = b-  Ajc  ^0 

X £ 

X £ 

These  three  QP  algorithms  are  primal  methods,  i.e.,  they  work  with 
feasible  solutions  of  the  problem  (l).  Generally,  they  require  an  initial- 
ization stop  which  obtains  a,n  extreme  point  of  £,  or — what  is  the  same 
thing — a basic  feasible  solution  of  the  system 

I V + A X = b (17) 

X ^ £,  V ^ £ 

If  the  initialization  step  indicates  the  constraints  are  inconsistent,  no 
computation  Is  required.  Otherwise,  a pivotal  transformation  of 
obtained  in  which  the  vector  on  right-hand  side  of  the  equation  is 
tive.  Thus,  the  assumption  that  b > C is  tantamo'ont  to  the  assump- 
i^i.n  that  an  initialization  procedure  has  already  been  applied  and  shown 
that  the  problem  is  feasible. 


k.2.  Beale's  Method.  One  of  the  earliest  and  most  successfully  developed 
algorithms  for  QP  is  due  to  Beale  [l],  [P],  [3].  Tiie  method  proceeds 
from  a basic  solution  of  the  linear  constraints  (l7)  expressed  in  the  form 

V = b - A X (18) 

X ^ £,  V ^ £ 

where  b ^ £.  This  formulation  makes  it  possible  to  express  the  objective 
function  in  terms  of  the  nonbasic  (independent)  variables  x,.  Furthermore, 
^f(x)  = X + ^ ^ can  be  written  in  the  form 


tfCx)  = H 


X 

O 


c H I X! 


- I l-J 


At  the  basic  feasible  solution  (v,x)  = (b,£)  the  gradient  of  is  given  by 
c,  and  it  follows  that  if"  b > £ (nondegoncrucy  assamption)  the  value  of  Cp 
can  be  decreased  locally  if  and  only  if  £ contains  a negative  component. 


! 

i 


1 
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If,  to  be  Kpocifie,  = min  ^ < 0,  then  an  iaiTC-ufii'  of  the  tionbasie 

va.rinb.le  x^,  will  brinp  about  a dec I'i a:.o  of  • ne  ob.V.-tive  fuTictirn  valu", 
at  leant  locaily.  in  term-n  of  the  initial  data,  the  variation  of  the  ba- 
sic variables  is  itoverncd  by  the  equation 


^1  " W 


a,  X 
■ is  s 


The  increase  of  the  variable  is  to  be  stopped  as  soon  as  one  of  the  ba 


i = 1 , . . . ,m 

pped  as  soon 

sic  variables  decreases  to  zero  (as  in  I,P)  or  the  partial  derivative  3<f/t)x 
increases  to  zero.  If  neither  of  these  occurs,  then  the  objective  function 
has  no  finite  lower  bound  on  £ and  the  procedure  is  terminated.  If  the  in- 
crease of  x^  is  first  blocked  by  v^  reaching  zero,  then  a new  basic  solu- 
tion is  obtained  ns  in  the  simplex  method  for  LP.  But,  if  the  partial  de- 
rivative vanishes  first,  then  one  tries  to  keep  that  partial  derivative  at 
the  value  zero  (at  least  until  sucli  time  a.s  it  is  found  advantageous  to  do 
otherwise).  This  is  accomplished  by  Introducing  a variable 


(19) 


which  is  defined  by  the  expression  that  gives  the  value  of  the  partial  de- 
rivative 3*P/9x^.  Tills  variable  is  said  to  be  free  in  the  sense  that  it  is 
not  sign  restricted.  In  order  to  ensure  tliat  the  partial  derivative  atf/ax^ 
remains  zero,  the  equation  (19)  is  adjoined  to  (l8)  thereby  yielding  m + 1 
equations  in  m + n + 1 unknowns.  In  this  enlarged  system,  the  variable  u^ 
is  made  nonbasic  in  place  of  (as  in  LP).  As  before,  there  are  n no.nba- 
sic  variables:  u^  and  the  Xj  (j  ^ s).  The  objective  function  can  be  re- 
presented in  tei-ms  of  the  new  set  of  nonbasic  variables. 

The  procedure  is  repeated  (with  ilie  new  system)  but  with  some 
special  rules  which  come  about  due  to  the  presence  of  free  variables  which 
by  definition  are  allowed  to  negative  as  well  as  positive.  If  there  is  a 
possibility  of  decreasing  the  objective  fijnction  by  increasing  or  decreas- 
ing a free  variable,  this  should  be  done — subject  to  the  same  principles 
that  limit  the  variation  of  any  nonbasic  variable — before  a sign-restricted 
variable  is  chosen  for  increase.  When  a free  variable  becomes  basic,  that 
variable  and  the  corresponding  equation  are  dropped  from  the  problem  be- 
cause retaining  them  no  longer  serves  a purpose. 

Termination  of  the  method  occurs  when  either  the  nonbasic  variable 
being  increased  (or  decreased)  from  zero  i .s  not  blocked  by  the  vanishing 
of  a basic  variable  or  a partial  derivative  or  else  there  is  no  longer  a 
way  to  decrease  the  objective  function  by  modifying  a nonbasic  variable. 

Ihis  description  of  Beale's  metiiod  is,  of  coui'se,  quite  brief.  In 
[1  ],  [?],  [3],  f5l,  and  [33]  ti.c  reader  will  find  m;uiy  more  details. 
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W'' LTf ' IT.  r.;.'',  ox  V.fMo.!  f'>>'  In  [^Bl,  V.'oi  . ■ nyiO'j<.-'i  1. /»•  t>ir-  jin- 

plr-x  nx'tJiod  of  liF  could  f'?  pul  to  ur.c-  in  i-iuivc’/  C;I'  ].;■-. nlui. . iiis 

mcLhod  cm.  be  r--7:m  do'i  a.;  a n-'i . m .i-tri  c C,?  piu-cedurt . (r.i/e  Section  C 

for  more  on  this  sort  of  pi'obJ  en.  ) It  is  deni ^Tied  to  solve  jirobleins  of  the 


for.T) 


nunitai'.'e  = > c_  x t-  'n  ^ ]i  x 

subject  to  A X 4 b 

£ ^ 0 

for  all  A ^ 0.,  Ibe  matrix  H is  asstitied  to  be  positive  semi -dtfini  te;  thus 
is  a convex  function  for  ell  A. 

Actually,  Wolfe  gave  too  procedures  which  he  called  tl.'O  t hort  form 
and  the  long  form.  In  the  short  form,  A is  fixed,  whereas  in  the  long 
form  it  varies,  thereby  making  it  a parair:etric  raeth.od. 

The  short  form  is  valid  for  problems  in  which  either  ^ is  positive 
definite  or  the  linear  term  A ~ jc  vanishes  identically.  The  latter  occurs 
when  A = 0 or  £ = £.  It  is  initiated  from  a solution  of  the  problem  in 
wlilch  A = 0 and  to  which  the  sliort  form  applies. 

The  Short  Form.  Consider  the  system 

H}c  + Ay-£  + £'-£"  = ~A£  (20) 


A 2c  + V = b 

2i  i H.  X ^£>  2.  £>  ^--9. 

Initially,  let  x,  jr,  and  u be  zero.  Select  a basic  feasible  solution  for 

the  system  (20)  from  among  the  variables  in  the  vectors  £,  z_'  and  In 

particular,  use  the  vector  v (it  being  assumed  that  b and  if  - Ac  '^0 

J 

use  7.!;  otherwise,  use  z" . Let,  z_  denote  the  vector  of  n basic  variables 
J J 

chosen  from  the  zj  and  zj',  and  let  E denote  the  corresponding  coefficient 
matrix.  Now  consider  tl;e  LP 

minimi.ze  £ £ 

subject  to  H X.  A y - u + E £ = - A £ 

A X + V = b 

21  }L  ra£j  H t«£>  Z i£’  i 

where  ^ = (l,...,l).  Tlie  algorithm  culls  for  the  solution  of  this  LP  by 

the  simplex  method  with  a restricted  basis  e?itry  ml e : if  u is  basic,  do 

J 

not  allow  X.  to  be  basic  (and  vice  versa);  if  v.  is  basic,  do  not  allow  y. 

J ^ * 

to  be  basic  (and  vice  versa).  The  minimuin  value  of  the  objective  function 

(tiie  sum  of  the  artificial  variables  z^)  must  be  zero.  An  optimal  solution 

of  this  LP  obtained  in  the  manner  indicated  will  yield  a solution  of  the 

Kuhn-Tucker  condition.",  for  the  QP. 

The  Lotir  Fo^-m.  The  possibility  of  solving  the  parametric  problem 


9 


vhen  X = 0 is  punraraft  d by  the  short.  I’orn  ' V/oift's  :iiet  hod . 'liie  sojutien 
of  the  I'araatetric  probie-n  via  the  long  forn  te.'ln.-.  wltii  the  solution  of  the 
Qi’  corresponding  to  X = 0.  Oticc  this  is  in  hona,  one  considers  the  LP 
maximize  ti 

subject  to  ii2t  + A^-^4p£-t_K.t-0 

A 21  t V = b 

21  H = £.  ju  2l  0,  V > 0,  £ -1  p ^ 0 

and  starts  from  the  solution  (giver;  by  the  short  form)  in  which  ^ u = 0, 

V = 0,  ^ = £,  p = 0.  The  maximization  of  p is  performed  with  the  sim- 
plex method  and  the  same  restricted  basis  entry  rules  as  in  the  short  form, 
ftirthermore , no  ^-variable  is  permitted  to  beco.me  l.asic. 

1*.U.  van  de  Panne  and  Whiriston's  Simnlex  .Method  for  QP.  Still  another 
prominent  algorithm  for  convex  QP  is  due  to  van  de  i'anne  and  Wl.inston 
[^8].  [50).  Actually,  these  authors  proposed  several  algorith.ms  only  one 
of  which  will  be  discussed  here,  namely  their  symmetric  prim.-'.l  simplex 
method.  A limited  form  of  their  "asjTimietric  niethcd"  was  euiticix.<'’.ted  by 
Dantzig  [l8],[l9]. 

Tlie  van  de  Panne-Whinston  symmetric  primal  simplex  met  od  provides 
an  appropriate  transition  to  the  study  of  the  linear  complementarity  prob- 
lem. Accordingly,  it  is  described  in  somewhat  more  detail  than  tlie  methods 
of  Peale  and  Wolfe.  The  method  uses  first-order  information  on  the  objec- 
tive fu-nctio.n  ^ to  guide  its  steps.  In  this  connection,  tiie  following  ob- 
servation is  important: 

Theorem  9-  For  any  solution  of  the  equations 

r-> 

u = £+  Hx  + Ay 


v = b - A X 


it  follows  that 


Tlie  lel^-hari'i  side  of  (21)  is  just  the  value  of  the  quadratic 
objective  function  ^(x)*  Theorem  9 has  the  further  consequence  that  if 
21  il  ” Z .Y.  “ 0,  then  the  vttlue  of  t \x)  is  given  by  the  linear  expression 

H (£21  -%y)- 

In  order  to  give  a reasonable  descii  pti^n  of  the  algorithm,  it  will 
be  helpful  to  consider  the  following  change  of  notitlon.  Let  Xj  - x, 

Xj  = V,  Ht  ~ £>  v.  = y.  The  letters  T and  J stand  for  sets  of 

indices.  Also,  let  6 = 2 ^ “ 21p  Zp  “ Yj  Initially  I = {!,..., n) 

and  J = fn+1 , . . . ,n+m) , but  these  sets  may  change  a.s  the  algorithm  evolves. 
In  the  new  notation,  x.  and  x.  are  the  priina]  variables.  Tnose  indexed  by 

— .J  — ^ 


I 


(2?) 


1 tiV'.*  noriVaoif,  vriij'j-  i ri.?' >.ctl  'py  <’  bupi  -. 

In  tnboJn;'  yor.'M  Mif;  '-V-int,  oqn-ilior.  : i ..-  r.c'.hoo  ■-■i 


1 



<v 

0 

0 

— t' 

eNy 

y 

c* 

A 

•- 

s 

X r 

-A 

0 

•-*v 



Thp  method  :>,r.oi -.ts  >J'  a pcqaencc  of  pivot  steps;  each  is  one  of 
the  follovinp  three  types. 

(i)  An  exch^mgp  of  tunic  y^  •■j’.d  nonh.usi;:  . 'index  s is  then  tren.sfe.’- 

red  from  I to  -/.) 

(jj)  An  exohanc;e  r.f  basic  and  nonbo'ic  y . (index  t i.s  then  transfer- 

V U 

red  from  J to  I .) 

(iii)  All  exchange  of  basic  x,  ui'd  nonbasic  x followed  by  an  exchange  of 

i/  r> 

basic  y and  nonrasic  y. . (index  a is  then  transfeired  from  I to  J 

li  t 

and  index  t is  transferred  from  J to  J.) 

Pivot  steps  of  these  three  typos  preserve  a structural  property  of  the  tab- 
leau known  as  bis^emnc.-try . Indeed,  let  the  row  corresponding  to  the  objec- 
tive function  and  the  column  of  constants  headed  (headed  by  1)  be  associat- 
ed with  the  index  0.  Then  define  = (o)  I.  Wext,  regard  the  entries 
in  the  tableau  as  the  elcmcntt  of  a matrix,  say  M.  Then  the  principal 
sutn.uLricos  M,-  , a-f-'i  r r sys-metric  while  M-  , is  the  negative  trans- 

~Vo  -^0'- 

poso  of  M.^  . Sue),  a corjhination  of  syronetry  and  skew-symnetry  is  called 
0 

hi  ryiTunetry . Th-.  ini'ial  tableau  (?!')  exl.ibits  thin  property.  An  extreme- 
ly important  feature  of  the  van  de  rtLnne-W.hi  nston  algorithm  i.s  that  the 
pivot  steps  it  uses  load  to  another  bisyrjr.etric  tableau.  Mere  generally, 
cne  can  state  [29) 

Hieorerr.  10.  Bisyi.'met-y  i.'’  preserved  by  principal  pivoting. 

IIiuT , at  any  stage  of  the  .algorithm,  the  current  tableau  can  be 
writtc-n  in  the  form 

(23) 


1 

^ 

e 

k 



£ 

-b' 

c ' 

H" 

r-* 

K 

b-* 

~ 

-A" 

'k' 

I an  d J 

in  (23) 

may  be 

The  husic  sol-.ition.s  en.countered  in  the  m.ethcl  all  sutlcf;.  the 


n 


r 


coiiiiitloti  • V , = 0 SO  that  0 = ? ^ (x,).  The  i.i"' ho^i  i;-;  Ir.lti- 

ateJ  with  a fa::ic  fea;;ible  solution  of  the  i'rir.al  r-,njtj-alr»t;  . i'liu;: , x,  = 

-T/ 

b > 0^  when  Xj  = 0^  and  = 0.  !lence,  It  renaiii.;  to  achieve  the  nonnega- 
tivity  of  . 

Just  as  in  the  simpiex  method  for  LP,  tl;e  van  de  Inniie-VvTiinsi.on 
algorithm  6;ntails  the  increase  of  one  nonbasic  variable  at  a time.  Tills 
will  be  called  the  driving  variable.  In  this  algorithm,  the  driving  vari- 
abie  is  always  an  x-variable,  say  , for  which  y ^ < 0.  The  increase  of 
the  driving  variable  is  blocked  if  and  when  the  value  of  any  basic  variable 
belonging  to  a particular  set  reaches  zero.  In  the  present  algorithm,  this 
set  consists  of  all  basic  x-variables  (which  are  nonnegative)  and  one 
particular  basic  y-variable,  y_, , whose  value  is  negative,  i'reservi.  g the 
nonnegativity  of  the  x-variables  means  the  same  thing  as  preserving  pri.mal 
feasibility.  Tlie  algor i‘ hm,  then,  goes  as  follows: 

Step  1.  Test  for  optimality.  If  = £ ^ 0_,  terminate.  The  current  solu- 
tion is  optimal.  Otherwise,  let  c <0. 

s 

step  2.  If  = 0,  go  to  step  3.  Otherwise,  use  as  the  driving  vari- 
able and  determine  the  blocking  variable. 

(a)  If  y^  is  the  blocking  variable,  perform  a pivot  of  type  (i),  and 
return  to  step  1. 

(b)  If  blocl.s  Xg,  and  if  0^^  > 0,  perform  a pivot  of  type  (li)  and 
repeat  step  2 with  as  the  driving  variable. 

Step  3.  Determine  whether  x is  blocked.  If  -A  >0  for  all  r ^ J,  ter- 
minate.  The  driving  variable  is  unblocked  and  the  objective  function  is 
unbounded  below  on  the  primal  constraint  set.  Othei'wise,  let  block  x^ . 

(a)  If  > 0,  perform  a pi\'Ot  of  type  (ii),  and  return  to  step  2 
with  Xg  as  the  driving  variable. 

(b)  If  0^^  = 0,  perform  a pivot  of  type  (iii),  and  retuj’n  to  step  1. 

Each  return  to  stop  1 that  does  not  lead  to  termination  specifies 
a distinguished  index  s ^ I such  that  c^  < 0.  Between  such  occurrences, 
the  st-t  of  eligible  blocking  varl ablcs  is  composed  of  y^  and  all  the  ba- 
sic primal  variables,  i.e.,  those  of  the  vector 

TTie  reader  may  notice  that  no  provision  is  made  for  the  possibil- 
ity of  negative  entrie.  appearing  as  diagonal  elements  of  ii  or  Q..  ITiis  can 
be  Justified  by  showing  that  the  positive  semi-definiteness  of  these  ma- 
trices is  preserved  by  the  pivotal  operations  of  this  method.  The  general 
result  along  tliece  lines  is  given  in  the  next  section.  Keller  (29}  has  ex- 
tended this  TK-thod  to  '’ind  a local  mln1na"i  in  a nonconvex  QP. 

1? 
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5.1.  Formulation . '.ilvon  .a  iiii:LiJX  M t and  a .f.-iaor  ^ G r:.f?  linear 

complementurity  iirobie;!'  is  tliut  cf  I'indir.g  a ;:olutio:i  to  ‘he  r.ysteii; 

V i + K z.  (2h/ 

ii  il>  2 > p.  (25) 

(2C-; 

(or  showing  tha.+  no  solution  exists),  ‘Hie  LCP  is  stated  in  term.s  of  CK 
variables  w . . . ,w  and  z,  For  each  i ~ 1,...,II  the  variaLle.s  w, 

X i Hi  j 

and  z^  constitute  a '-.mplomeiitary  T2i_Lr_  and  each  is  the  comnle.Tfr.t  of  the 
oth'-T.  Condi  tU'.s  (2?)  ai'-d  (2b)  impl;-  that  for  eac'n  i 

z^  w.  - 0 f?7) 

This  is  the  familiar  c-OEplernentary  nlackners  property. 

There  is  a nice  geometrical  interpretation  of  the  LCr.  If  (2h) 
and  (25)  have  a solution,  then  3 belongs  to  the  convex  cone  spanned  by  the 
nonnegative  linear  combinations  of  the  columns  of  the  matrix  I X>  " M ^ • 
iTic  LCP  asks  to  ha\e  3 so  represented  but  in  such  a vjay  that  for  all  i,  not 
both  the  i-th  coluim  of  ^ and  the  i-th  column  of  - M are  used.  A matrix  B 
i of  order  N such  th.at  for  each  1:  B^.  S (I.,  --  M.  } is  called  comrlementar; 

with  respect  to  M.  For  a problem  of  order  II,  there  are  2 ways  to  select 
the  matrix  B,  although  the  choices  need  not  all  be  distinct.  The  convex 
cones  they  span  are  called  eomplementaipz  cones . The  problem  (a,M)  can  then 
be  construed  as  that  of  determining  whether  belongs  to  some  complementary 
cones,  and,  if  so,  which  ono(.s).  An  actual  solution  w,z_  of  the  problem  is 
given  in  terr.is  of  the  weights  used  on  ll.e  coluirns  of  a complementary  matrix 
B. 


5.2.  Exampl es . Linear  complementarity  problems  arise  in  a number  of  ways, 
the  most  common  being  as  Kuhn-Tucker  systems  (ll)  for  quadratic  programs. 
Tne  formul.ation  of  (ll)  as  a linear  co.T.plementarity  problem  is  accomplished 
by  defining 


H T 

o')  f u'l 

-A  0 

An  LCP  of  this  type  is  special,  if  only  by  virtue  of  t.ne  bisyiumetry  of  M. 

In  i.he  convex  QP  case,  the  matrix  M (dcfii'.cd  above)  is  positive  stnvi-defi- 
nlte  in  the  sense  that 

i!  .'1  L P ® ^ L ^ i' ' (P8) 

Thi.s  condition  m.ake.s  .sense,  ever,  if  M is  not  symm  trie.  Moreover,  it  is 
very  useful  to  look  at  positive  semi -definl teness  this  way. 


^ 
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Not  L-vory  I,Ci’  .iri  3ca  as  a Kuhn-Tiickf. r sysvi-tii,  oven  thou;  *i  it  " '17 
in  fact  be  oiie.  This  is  t)ie  case  vhcn  th'-  hCP  rr,;ir.,.-cr,tL;  c^iu;  1 ' *ir i un  ccri- 
ditioiis  for  u inochani  cal  (or  eccno.'d  c ) system  [37],  [30].  -It  i-i^st  subse- 
quently be  discovered  that  these  equilibri  u-m  conditions  arc  tla  Kuhri-Tucker 
conditions  for  a QP.  tinder  these  conditions,  alternate  rolntion  strategies 
may  su^'.f.ost  themselves.  That  is,  one  may  iiave  a preference  (possibjy  based 
on  the  availability  of  a software  paokape)  for  solving  tiie  LCl’  as  the  QP 
whose  Kuhn-Tucker  conditions  it  reprcsentr. . 

However,  this  approach  is  not  valid  when  Die  matrix  M lacks  bisym- 
metry,  and  since  there  exist  linear  complementarity  problemis  where  this 
property  is  absent  [l3],  [ 3^4  ] , it  is  fortunate  that  the  theory  of  linear 
complementarity  does  not  rely  on  tlie  bisyiir.-.otiy  condition.  In  this  respect 
the  LCP  is  more  general  than  the  QP. 

5-3.  Existence  and  Uniqueness  Results.  Wlien  a problem  (ti,M)  is  presented, 
it  is  very  helpful  to  know  what  kind  of  a matrix  M is.  Describing  the  full 
range  of  interesting  possibilities  would  be  inappropriate  here,  but  the 
theorems  quoted  in  this  section  cover  the  most  important  cases  and  should 
give  an  indicatio.n  of  how  M can  influence  the  qualitative  properties  of  the 
probl em . 

I 

The  matrix  M belongs  to  the  class  P if  and  only  if  all  principal 
minors  of  M are  positive,  i.e., 

det  >0  for  all  C IJ}  (29) 

Members  of  the  class  P are  called  P-matrices.  Note  that  because  M is  not 
assur:;t'd  to  be  symmetric,  condition  (29)  is  not  equivalent  to  positive  defi- 
niteness. It  is  a fact,  however,  that  if  _M  is  positive  definite  in  the 
sense  that  £ M ^ > 0 for  all  ^ ^ 0_,  then  M ' P 

Theorem  11.  The  LCP  (ci,M)  has  a unique  s ution  for  every  n.  ^ if  and 
only  if  M ^ P- 

This  result,  due  essentially  to  Snmclson,  Thrall  and  Wesler  [5l<J, 

says  (in  the  language  of  Section  5-1)  that  the  complementary  cones  relative 

N 

to  M actually  partition  ^ . 

A second  result,  one  that  is  closely  related  to  applications  In 
engineering  plasticity,, concerns  positive  seirl-definiteness . It  was  first 
proved  by  Cottle  [ 9 1 

Theorem  12.  If  M is  positive  semi-definite  and  the  system  (2!0,(?5)  has  a 
solution, ttien  (j.,M)  has  a solution. 

In  the  positive  .semi -definite  case,  neither  the  existence  nor  the 
3 of  a solution  is  guaranteed.  However,  when  (£,M)  Is  nondegen- 

II4 


unlqucno  > 


(.•ret’'  i:  , -'vory  mOu!.  cr  (?h)  ■ 'I  hi  r.i  ■ . i 

aiid  M i;:  1 ' vi-  rn-mi -•'<  ;'i  ;:i  I-:- , Ujfti  o:  ■■  ' i.  [•'!,]. 

Th'/  fu.l  ) o'..i  !j('  ) :<  a ,i,n.-.l  I'i."'  a ■ J . ci-.i  ^ , ’’y  ,■  ro- 

)<j  CTi.s  yh.’w;;  t>ic-y  )iavo  r;^;ae' h i a^.•.  ;:i<  m.-  lin-:r  c ''nal  j o ! i.‘ a ?a  c-jiaioii 

vitl'  U’  prOLlc-na. 

'J'heorcin  .13-  IT  Uic  I.C>'  (a,^)  lio.f.  u sol  .',ian,  i*  I,',:  a ■■.aiu*.  ; ..rrich  i:.  sa 
fxtrc-;:c  j'oint  of  tho  fict  ■ li  " li  ' !1  ii>  i!.  P.v  1 ? I*)- 

ITiia  oLservaticn  has  pi > n rise  t-j  '.hf  of  soil  vi  ac,  Jinoai- 

comploir.ontarity  [-roblors  via  prcp,ra..,.'ii;i.  ' o,  ; ni aues  . Thi ui  j '"oach 

raqi.lrc’.  thf  a pjiovi  idoiuif:’  •.oKii:  of  a Ji.iour  1'  ra  that  will  be-  iiiiai- 
iriy.ou  at  n co/ijv:  f!i-.ef.*-a;-y  c-xirv.-'i.-  point  of  tho  "fe;.;.  ibl  o set.”  Alt’f.wpii  thi- 
sub.li-ct  is  far  too  tf-aiinica'i  for  i nci  aoi or.  ir.  t'-.j ..  j -aj.iio  tne  i-cader  i.r  eii- 
cour;:y,ed  to  consult  1 39  i , f'lO],  [111],  [15],  [i^>]  for  a prese.atatioii  of  the 
idea  and  its  siprufican-'e.  0i,o  si.op.le  instance-  vbert:  the  LP  approaoJi  is 

applicable  warraiits  attention  hei-o. 

Nxll 

Tlic  natrix  W C h‘  bt-lonps  to  tlie  class  ^ if  and  only  if  all  of 
its  of f-difir,onal  entri  c-s  (M.  , , i ?!  J ) are  nonposi  ti  vo  . b'lernenta  of  Z are 
called  ^-matrices  [23].  'Plio  follcwinp  is  due  to  Cottlo  and  '.’oinott  ]17]. 
Theorem  ll|.  If  M G , the  following  are  eriui  valent: 

(i)  for  each  ££11",  the  set  ^(£,M)  contain.^  a least  eleiiient  (that  is, 
an  element  satisfyinf.  j;  £ for  all  ^ ) snd  £*  is  the 

unique  element  of  ^(a,H)  satisfying  £ (£  ” " ~ 

(ii  ) M e £ n b 

notice  that  whenever  a polyhedral  set  C contains  a least  element, 
that  eierrent  minimizes  any  pcsitivi-  lin  -ar  form  of  the  set.  In  the  case  of 
an  LCP  (q,")  where  M £ P £'i:i  unique  solution  is  tlie  least  element  of 
Z'£,M)  and  perforce  solves  the  I,P 

minirri  ze  ££ 
sub.ieet  to  M 3. 

where  ? = (1,.  . . ,1)  £ 


5. It.  Pivotal  Algebra.  The  method  described  in  'J..ctlo:,  5-5  an  i i-tn.-rs  de- 
pends }>cavily  on  some  algebriic  J'e.sults.  Oiii'  of  (V.  ,-  i.-  -wi.!l  1.- 
presented  licre  in  order  to  give  r.-;ore  meaning  to  wi.-.t  is  dom  in  trie  ri-iiuc-l  . 

Con.sider  the  case  where  the  i npri-,;;  ,nts  of  the  eiui’ici  {.’l|l  ire 
partitioned  as  follows 

Vj  = ^ Zj  ^ _M 

ti,,r  if  * tv.’  V 


! 


mm 


'I’he  subscripts  / anJ  ,!  represent  complementary  subsets  of  {1,...,IJ), 
possible  to  solve  for  in  terms  of  and  provided  exists, 
does,  and  the  exchange  is  execute-d,  the  equivalefit  syste.m 


It  is 
If  it 


- M, 


-II  -I 


- m7^  r;„  7.. 

—II  —IJ  -V 


results . 


-J  -II 

»,  • 3.,  - ij  • * (>■,.,  - Sji  <£u  ij 

The  coefficient  matrix 

T _ T 

M, 


kv 

- 

-II 

iM  M ^ 

-II 

- 

-l/c' 

trfuisfcrm 

of 

M = 

Cm 

-11 

M 

-IJ 

M 

and  the  process  is  called  principal  transfor.oiation  or  principal  pivoting. 

Tlie  next  theorem  summarizes  four  separate  results  on  principal  pjivoting. 

NxN 

Theorem  15.  het  M'  6 R be  a principal  transform  of  M. 

(i)  If  M is  a P-matrix,  so  is  M ' . See  [57 1 • 

(ii)  If  M is  positive  definite,  so  is  M'.  See  [lo]. 

(iii)  If  M is  positive  semi-definite,  so  is  M ' . See  [lo]. 

(iv)  If  M it',  bisymmetric,  so  is  M'.  See  [29]. 

This  theorem  implies  that  when  an  algorithni  calls  for  a principal 
transformation  of  the  system  (2!»),  and  M has  any  of  the  properties  men- 
tioned in  (i)  - (iv),  it  will  be  preserved.  Parts  (iii)  and  (iv)  of  this 
tlieorem  are  used  in  Justifying  the  vari  de  I’anne-V.'hlnston  algorithm  pre- 
sented in  Section 


5.5.  The  Principal  Pivoting  Method.  In  the  case  where  M ^ P,  the  LCP 
(q,M)  can  be  solved  for  any  9.  G R . One  particular  method  for  doing  this 
is  called  the  principal  pivoting  method  [ ?0] , [9],  [ 12] . In  contrast  to 
the  quadratic  progranm.ing  methods  discussed  In  Section  4,  it  doer,  not  use 
an  objective  function  to  guide  the  pivot  steps.  Instead,  it  consists  of  a 
sequence  of  major  cycles  each  of  which  is  associated  with  a distinguished 
baric  variable  that  happens  to  be  negative  in  value.  Tlie  major  cycle,  in 
turn,  consists  of  a sequence  of  principal  pivot  steps  tliat  (i)  prevent  al- 
ready nonnegative  basic  variables  from  becoming  negative,  and  (ii)  termin- 
ate when  the  distinguished  variable  increases  to  zero.  The  complement  of 
the  distinguished  variable  is  nonhasic  and  acts  as  the  driving  v.arlable 
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thr.'Miphout  the  r.ajor  ryolo.  The  variable"  that  are  ellgibie  to  block  ite 
increase  are  t)ie  dlstinguirhed  variablj  anti  all  the  nonnegativ.j  basic  vari- 
ables. This  distinguishes  the  principal  pivoting  method  from  the  van  de 
Panne-Whinston  which  in  some  other  respects  is  similar. 

The  system  (2h)  can  be  represented  in  the  tabular  form 


The  algorithms  can  be  stated  as  follows: 

Step  0.  Start  with  the  basic  solution  (w,^)  = (^,0_). 

Step  1.  If  2.»  s'top*  Th®  solution  is  at  hand.  Otherwise,  select  an 

index  s such  that  q < 0.  (This  might,  but  need  not,  be  chosed  so  that  q 
is  the  smallest  component  of  c^. ) Designate  w as  the  distinguished  vari- 
able  and  as  the  driving  variable. 

Step  2.  Determine  which  variable  blocks  the  driving  variable.  This  Is 
done  by  calculating 

“ ^ < 0». 

Step  3.  Perform  a principal  pivot.  If  t = s (l.e.,  the  blocking  variable 
is  the  distinguished  variable)  then  exchange  w and  z ; return  to  Step  1. 
If  t 3,  hold  Zg  at  the  value  - and  perform  a principal  pivot,  ex- 

changing w^  and  z^.  Return  to  Step  2. 

It  should  be  pointed  out  that  some  refinements  are  required  to 
keep  track  of  the  true  Identity  of  the  basic  and  nonbasic  variables.  As 
the  algorithm  is  stated  above,  the  basic  variables  are  denoted  generlcally 
as  w's  and  the  nonbasic  variables  as  z's. 

Another  Important  theoretical  point  is  that  to  be  perfectly  cor- 
rect, it  is  necessary  to  consider  the  problem  of  degeneracy.  This  can  be 
done  by  Introducing  suitable  perturbations  or  lexicographic  ordering.  It 
can  also  be  done  by  implementing  certain  tie-breaking  rules.  See  [22). 

There  are  other  principal  pivoting  methods,  and  other  classes  of 
problems  to  which  they  apply — most  notably,  the  positive  semi-definite 
case.  See  [27],  [12],  [13).  [UM,  [>»5]. 
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5.6.  iK-rr.kc'r.  j.^olhod.  Ano-ther,  wnre  rcbur.t,  method  for  the  hCJ’  is  due  to 

[ 3**  K [ 35 1 • There  ure  different  ver;aons  of  the  same  underl/ln^?,  ide<r 
but  regrettably  there  is  only  roc.;n  in  this  j upon-  I'ur  one  of  them. 

Consider  the  auxiliary  LCP 


(29) 

^ L h P-*  ^0  i 

(30) 

^ w = 0 

(31) 

A solution  of  this  system  with  Zq  = 0 Is  necessarily  a solution  of  (£,M) . 

If  £ is  a positive  vector,  then  for  £ = 0 and  a suitably  large  value  of  Zq, 
the  vector  w = £ + £ Zq  ^ 0,  so  (29)  and  (31)  hold.  But  if  > 0 holds, 

then  (2U)  does  not.  The  Lemke  method  preserves  properties  (29),  (39),  and 
(31)  and  works  by  means  of  special  pivot  selection  rules  to  achieve  the 
condition  Zq  = 0.  The  system  (29)  can  be  represented  by  the  tableau 


*1  •••• 

''l 

• « 

Mil 

Min 

''n 

% ®N 

«N1  ••• 

Mnn 

Lesike's  scheme  goes  as  follows. 

Stop  0.  Start  with  the  basic  solution  (w,Zq,£)  = (£,0,0). 

Step  1.  If  £ ^ stop.  The  solution  is  at  hand.  Otherwise,  let  the  in- 
dex .s  be  defined  by  the  condition 

- = max^  { - q^/e.) 

Perform  a pivot  step  making  z_  basic  in  place  of  w . (Now  both  w and  z 

(J  5 8 3 

are  nonbasic.)  Designate  z^  as  the  driving  variable. 

Step  2.  Determine  whether  the  driving  variable  is  blocked,  and  if  so,  by 

which  variable.  This  is  done  by  checking  whether  the  column  vector  (in  the 

current  tableau)  associated  with  the  driving  variable  z^  is  nonnegative. 

If  it  is,  then  z^  is  unblocked,  and  the  procedure  is  terminated.  Otherwise 

the  increase  of  z is  blocked  by  some  basic  variable  which  decreases  to 
s 

zero  first. 

Step  3.  Perform  a change  of  basis.  Exchange  tlie  (basic)  blocking  variable 
and  t)ie  (nonbasic)  driving  variable.  If  the  blocking  variable  v/as  z^,  stop. 
A solution  is  at,  hand.  If  not,  let  the  complement  of  the  blocking  variable 
(now  nonbasic)  be  the  new  driving  variable.  Return  to  Step  2. 

Remarks  similar  to  those  of  Section  5.5  must  be  made  shout  the 
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ntv'i  for  handliin.’;  in  thi:;  C'f't.i.O'i.  (S'-’O  haven  [2?]  on  this  sub- 

Jec'..)  But  equally  important  is  tbe  question  of  ;.l.ut  can  be  deduced  from 
the  lack  of  a blocking  variable  in  Step  2.  This  occurrence  triggers  ter- 
mination of  the  procedure  ( on _a  2'ny  as  it  is  often  called),  and  it  raises 
the  question  of  what  it  means  in  terms  of  solvability  of  the  original 
problem  (^,M) . Some  insight  can  be  derive'!  from  the  following  result  [34]. 

Theorem  l6 . If  the  Lemke  algorithm  ipplied  to  (fi,M)  termnates  on  a ray, 

there  exists  a norr-iero,  nonnegative  vector  u such  tliat  , 

(li  0 i = 1.  • ■ • >N  ( 32) 

In  the  case  of  P-matrices,  the  system  (32)  cannot  have  a nonzero 
solution,  and  consequently  Lemke ' s algorithm  solves  this  class  of  problems. 

When  M is  positive  semi  — lefinlto  (or,  more  generally,  copoaitive-plus) , 
termination  on  a ray  implies  that  (24)  and  (25)  ha-ze  no  solution.  While  it 
is  known  that  Lt.'mke's  method  can  be  applied  to  other  classes  of  linear  com- 
plementarity problems,  it  is  doubtful  that  the  full  extent  of  its  applica- 
bility has  yet  been  determined.  Investigations  along  these  lines  are  to 
be  found  in  [22],  [36],  [25],  among  others. 

6.  PARAMETRIC  I-tETHODS 

6.1.  Tntrodgeti -on . In  a number  of  instances,  one  is  interested  in  solv- 
ing a QP  or  LCP  for  all  values  of  certain  parameters  within  a specified 
range.  One  such  Is  exemplified  by  the  long  form  of  Wolfe's  simplex  method 

for  QF  considered  in  Section  4.3.  The  problem  treated  there  entails  a ' 

special  type  of  variation  of  the  linear  term  in  the  objective  function. 

In  the  case  of  Wolfe's  metliod,  the  para4iictr  j zation  is  as  much  nested  in 
the  method  for  solving  the  nonparametric  problem  as  it  is  in  the  formula- 
tion of  a truly  parametric  problem,  such  as  the  so-called  portfolio  selec- 
tion problem  to  which  it  is  manifestly  applicable.  See  Markowitz  [42]. 

For  applications  of  the  parametric  LCP,  see  Maier  [38]- 

6.2.  Param>*tric  QP  problems.  The  quadratic  program  (l)  is  .specified  by 
the  data  A,  b,  £,  H.  The  most  co.mmonly  encounterfd  parametric  versions  of 
the  QI’  problem  can  be  formulated  as  follows.  Let  A be  an  Interval  of  the 

real  line.  For  all  X S A,  j: 

minimize  (£  + X'^)  x + H.  x.  (33) 

subject  to  A i 4 k ^ 

X > 0 
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where  c^'  aiiJ  are  given  vectors.  I f = 0 and  £_'  f the  problc-a  has 
a parametric  rb^'iect  i ve  I'aiction,  ,;hile  If  h'  / 0 cuid  c'  = £,  the  prcbJem 
has  a para;r.etric  rignt-hand  side. 

Kor  a given  value  of  X G A,  there  is  a c'  rresporiding  QP  which  may 
or  may  not  be  feasible,  may  or  may  not  have  an  optimal  solution.  Since 
parametric  programming  methods  generally  proceed  from  an  optimal  solution 
cori'esponding  to  a particular  parameter  value,  there  is  a little  problem 
about  getting  started.  But,  in  the  convex  case,  tliis  is  not  terribly  dif- 
ficult. To  find  a value  of  X for  vnich  the  corresponding  QP  has  an  opti- 
mal solution,  it  suffices  to  write  down  the  conditions 

^ H i ^ ^ 2.  ( 3^ ) 

b+^^X-Ax  ^0^ 

ii  i 2>  i i 2>  ^ ^ A 

When  A is  an  interval,  testing  these  inequalities  for  a solution  is  the 
same  sort  of  problem  one  meets  in  LP.  If  a solution  (x®,,^®,X®)  is  fouiid, 
then  for  X = X®,  the  corresponding  QP  has  an  optimal  solution.  If  (3^) 
has  no  solution,  then  there  is  tio  value  of  X S A for  which  the  correspond- 
ing QP  has  an  optimal  solution.  This  is  a consequence  of  Tdieorem  8 (and 
of  Theorem  12).  Thus  the  procedure  can  be  made  to  start  from  X®. 

Among  the  types  of  questio.ns  that  parametric  QP  models  attempt  to 
ariswer  are 

(i)  How  does  the  optimal  value  vary  with  X? 

(ii)  How  does  the  optimal  solution  vary  with  X? 

(lii)  Over  what  range  of  values  of  X are  particular  sets  of  variables 
positive?  of  zero? 

Parametric  QP  of  the  type  described  above  can  be  treated  as  a parametric 
LCP.  For  the  sake  of  brevity,  this  approach  will  be  taken  here. 

N N NxN 

6.3.  Parametric  LCP  Problems.  Given  ^ ^ 2 > E ^ ^ 5.  > 

parametric  LCP  consists  of  solving  the  family  of  ordinary  LCP  problems 
+ A £,M)  for  all  A G A.  The  Kulm-Tucker  conditions  of  the  parametric 
QP  (33)  are  subsumed  in  this  formulation. 

In  the  discussion  below,  it  will  be  assumed  that  M is  either  a F- 
matrix  or  is  positive  sem: -definite . In  the  former  case,  (^  + A £,M)  has 
a unique  solution  for  each  A G H,  hence  for  each  A G A.  In  the  positive 
serai -definite  case,  the  problem  has  a solution  for  every  A for  which 

^+A£+M£^0,  £^0^  (35) 

has  a solution.  Accordingly,  in  order  to  get  started,  one  can  solve  the 
problem  of  determining  a A G A for  which  (35)  has  a solution.  If  there  is 
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nonf' , thfre  notlar.(;  r\.:-U)oi'  to  be  iloi:e. 

Lot  t.e  u ;'.ir:i:.,otcr  va'!  ue  for  v.’.icK  (e  > X*^  £>-1)  huo  a solution. 
One  can  asLiuno  that  X^  " 0.  (Oi.herwl /.e , iLi-  c):  ii.cc  os'  varlubU-,  9 = x - 
Pjivosrise  to  an  equivalent  pr'jbl  oir.  ((^  + X®  £)  + 0 £,^)  vhic)i  in  solvable 
for  0 ■=  t)-)  I'iius , v/ith  X^  = 0,  the  solution  procedure  for  (£1,^)  yields  a 
jirincipal  transforn  of  t!ie  prcblorn,  say  fc',M'),  iji  which  2.'  £2.*  Let  p' 
be  the  corresj.'onding  transfoj-j.-.at Ion  of  £.  Tlitio  + X £,M)  becomes 
(c^'  *■  X £' >11^ ) under  the  action  of  the  j)ivoting.  The  variation  of  X can 
now  begin. 

Without  loss  of  generality,  one  may  assuine  the  problem  is  to  be 
solved  for  f'J.1  X > 0.  If  negative  values  of  X are  to  be  considered,  one 
nay  simply  reverse  the  sign  of  £ , and  proceed  as  indicated  below.  If  A 
is  a bounded  interval,  the  increase  of  X can  be  stopped  at  an  appropriate 
point.  Notice  that  if  v > £,  the  problem  Is  trivial,  since  then  ^ + X £ 

is  nonnegative  for  all  X ^ 0. 

An  algorithm  is  therefore  needed  to  solve  the  following  problem. 

N N NxN 

Given  > £.^5,  > vhere  min  ^ 0 ^ min  ^ p^  , solve  the 

problem  (£  + X £>M)  for  all  X ^ 0. 

Recall  that  by  an  earlier  assumption,  M is  either  a P-matrix  or  a 
positive  semi-definite  matrix.  In  order  to  simplify  the  statement  of  the 
param.etric  LCP  algorithm  below,  it  will  be  assumed  that  o_  > £ and  for  each 
positive  value  of  the  parameter  X , at  most  one  basic  variable  vanishes. 

It  is  possible  to  dispense  with  this  nondcg.eneracy  assumption  as  in  [ll]. 

Since  > £,  it  follows  that  {jr,£'  = (£>£'  solves  (3.  + 0 £,M)  . 

That  is,  one  starts  with  a nonnegative  compleraentar-y  solution  of  (pA). 
the  parametric  algorithun,  one  increases  X (much  like  a driving  variable) 
and  maintains  a solution  of  the  LCP  {3^  + X £,■!)• 

Step  0.  Initialization.  Start  with  X = 0 and  (w>2^)  = (3,2).  Regard  X as 
the  driving  variable. 

Step  1.  Monotonicity  check.  If  the  driving  variable  is  unblocked,  stop. 
No  change  of  basis  is  required  for  larger  values  of  X, 

Step  2.  Determination  of  the  next  critical  value.  Determine  the  index  r 
by  the  condition 

- %/Py  = rain  ^ { - q^/p^  : < 0 ). 

The  driving  variable  X is  blocked  by  the  r-th  basic  variable  when  it 
reaches  the  critical  value  - q /p  . 

Step  3.  Change  of  basis. 

3.1.  If  = 0,  go  to  Step  3.2.  > 0.)  Pivot  on  holding  X 

at  tlie  current  critical  value.  Return  to  Step  1. 


21 


r 


3.?. 


If  M . *"  0 l\ir  an  i,  titop;  the  rrohlor.  haa  r.o  st)'u‘.ion  for 

J-i  =4  i i . 1 

larrer  values  of  A.  Otherwise  (since  V = 0,  M . > 0 for  soiue 

11'  n 

i and  M is  positive  a<  mi-definite)  det.ii  - ine  the  index  s by  the 


condition 


.1  ^s  - 1 i 

- r,' — ( q q ) - nd  n { — ( q ■ ) : f’h  < 0 }. 

M ^s  p ‘r  M.  1 p ^ ir 

sr  r ir  'r 

Then  perform  the  h.lock  pivot  on  tlic  nonsingu.lar  matrix 


M 

M 

rr 

rs 

M 

M 

sr 

ss 

Keturn  to  Step  .1 . 


There  is  a si.Tiple  geometric  interpretation  of  the  parametric  LCP 
and  the  algorithm  stated  above.  The  set  of  points  of  the  form  Q.  ^ £ for 
X ^ 0 is  a half-line  crManating  from  _q^  (corresponding  to  X O)  and  heading 
out  of  the  nonnegative  orthant  in  the  direction  £.  The  nonnegative 
orthant  is  a complementary  cone.  The  problem  is  to  mai.ntain  a represen- 
tation of  the  general  point  of  this  half-line,  i.e.,  ^ t X £,  as  a member 
of  a complementai*y  cone.  Clianges  of  basis  (i.e.,  representation)  occur 
when  for  a critical  value  of  X the  half-line  meets  the  boundary  of  a com- 
plementary cone.  Termination  of  the  algorithm  in  Step  1 means  no  more 
boundaries  will  be  met,  whereas  termination  in  Stop  3.2  means  that  for  any 
value  of  X larger  than  the  current  critical  value,  there  is  no  compleiaen- 
tary  cone  containing  + X £. 

In  some  applications,  it  is  desirable  to  kr.ow  whether  tlie  compo- 
nents in  a solution  are  nondecreasing  functions  of  X.  This  is  the  case 
if  and  only  if  the  coefficients  of  X in  the  rows  of  the  basic  z^  are  non- 
negntive.  It  may  also  be  of  interest  to  know  this  Information  in  advance 
of  the  actual  computation  [38];  this  question  is  investigated  in  [lO 1 . 


7.  COHTUTATTOIJAIi  KXPEHIEMCE 


Surprisingly  .little  of  a systematic  nature  is  known  about  the  rel- 
ative merits  of  quadratic  programming  and  linear  complementarity  methods 
and  tlieir  implementations-  in  software  packages.  Several  explanations  might 
be  offered  for  this,  not  the  least  of  which  would  be  the  sheer  expense  of 
the  undertaking;  tlie  develop.mcnt  of  one  truly  profession.al  calibre  code  is 
a -coctly  project,  and  the  comparison  of  methods  is  more  so.  Still  further, 
there  appears  to  be  no  body  of  recognized  test  problems  to  which  would-be 
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expcr  j n;ei'.ti:rs  cnn  apply  t!'.c-  various  ir.oUiod;;.  )iut  a few  raporl;-.  have  ap- 
pcari-d,  and  theae  are  hricfly  dc.  criteJ  hoiov. 


An  early  effort  at  co;r.pai'i;;on  was  ])ubliPhcd  by  van  de  ['nnne  and 
Vliinston  [*-9j.  Ihey  considered  a fcrii.ulnt  ion  of  llea.le's  method  with  their 
OV.T1  Simplex  I’iethod  for  Q?  (the  asyrir-ctr i c version)  and  concludes  tliat  the 
latter  is  superior  in  terms  of  iterations  and  storay.c.  This  pr  .'mpted  a 
spirited  response  from  Beale  [ )i  ] who  drew  attention  to  the  more  compact 
(practical)  version  of  his  leothod  presented  in  [3  ].  Beale  emphasized  two 
other  points.  One  is  that  the  Hessian  of  the  cpiadratic  objective  function 
lias  low  rank,  relative  to  the  number  of  variables,  and  the  practical  version 
of  his  method  takes  acivantat^e  of  this  fact.  The  second  is  that  the  practi- 
cal implementation  of  his  method  is  far  more  capable  of  handlinc  problems 
with  large  numbei-s  of  variables. 

At  about  the  same  time,  Moore  and  Vrhinston  [1(3 1 published  a compu- 
tational study  comparing  Volfe's  algorithm  with  those  of  the  Dantzig-van  de 
Panne-l'/hinston  family.  Wolfe's  method  seemed  less  attractive. 

A study  by  Lraitsch  [ 7 ] analyzed  the  methods  of  Dantzig  (van  de 
Panne  and  Whinston),  Beale,  and  Vlolfe  (in  the  original  version  and  a modi- 
fied one  due  to  Braitsch)  on  small  randomly  generated  problems.  He  pointed 
out  the  various  advantnges  of  the  different,  methods  already  mentioned  here 
and  seemed  to  indicate  the  slight  superiority  of  iiantzig's  method  over  the 
other.s . 

On  the  LCP  side,  some  encouraging  computational  experience  was  re- 
ported by  Bavindran  [53.]  who  experimented  with  a modification  of  Lemke's 
method  on  linear  prog'-arming  problems.  Bavindran  confirmed  that  this  adap- 
tation of  Lemi:e's  method  is  identical  to  the  so-called  self-dual  parametric 
algorithm  of  Dantzig  [19]-  He  also  reported  the  computational  .=;uperiority 
of  this  approach  over  the  primal  simplex  method.  Bavindran  has  published 
a computer  routine  for  Lemke's  method  in  its  general  form  [52]. 

AriOther,  quite  sopliisticated , version  of  Leni>;e's  method  has  been 
developed  by  TomJ.iii  [55],  [56].  His  code,  called  J^CPL,  maintains  a basis 
inverse  in  product  form  and  uses  LU  decomposition  to  reiiivert  it  at  a user- 
spccificd  frequency.  Computational  experience  privately  reported  by  users 
of  t^.is  program  suggeatb  that  it  performs  quite  satisfactorily.  A small 
amount  of  computational  experience  with  HUIlCMiiE,  the  forerunner  of  LCPL, 
is  recorded  in  [j-2  ] . To  this  may  be  added  the  following  table  which  gives 
merely  an  indication  of  tVic  program's  pcrfor?uance  on  a set  of  5 problems 
of  very  different  type.-. 
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I’robleni  N 


Jter 


Ti.-.c  (J?::-;  3'rj,  jt'.')) 


15 
36 
105 
520 
526 

It  is  important  to  reaiize  ti.at  although  some  methods  are  (theo- 
retically) applicable  to  aji_l_  memhers  of  a broad  class — e.g.,  convex  QP 
problems — there  are  occasions  when  special  methods  are  better  suited  for 
the  solution  of  problems  belonging  to  a particular  subclass.  Illustrations 
of  this  remark  are  to  be  found  in  t!ie  papers  [12  ],  | lb] , [15  ],  [26  ],  [28  ],  (lif4  ] , 
[^*5]  where  large,  sxiecial ly-structured  problems  are  attacked  by  specially 
desip/ied  methods  which  could  not  be  presented  here  for  reasons  of  length. 
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